Epsilon Entropy of Probability Distributions

This paper summarizes recent work on the theory of epsilon entropy for probability distributions on complete separable metric spaces. The theory was conceived [3] in order to have a framework for discussing the quality of data storage and transmission systems. The concept of data source was defined in [4] as a probabilistic metric space: a complete separable metric space together with a probability distribution under which open sets are measurable, so that the Borel sets are measurable. An a partition of such a space is a partition by measurable a sets, which, depending on context, can be sets of diameter at most e or sets of radius at most ie, that is, sets contained in spheres of radius 2e. The entropy H(U) of a partition U is the Shannon entropy of the probability of the distribution consisting of the measures of the sets of the partition. The (one shot) epsilon entropy of X with distribution A, He;,,(X), is defined by (1.1) He;p(X) = inf {H(U); U an c partition} U